Nested SoABlocks by Electricks94 · Pull Request #51554 · cms-sw/cmssw

Electricks94 · 2026-07-24T13:01:09Z

PR description:

The SOA_BLOCK macro used by SoABlocks accepts only SoALayouts that are not in itself an SoABlocks. If the Layout is an SoABlocks itself the macro fails with cryptic error messages. This PR extends SoABlocks so that it allows to add an SOA_BLOCK which is in itself an SoABlocks Layout. In this way nestes Layouts of Blocks can be formed.

Furthermore, the PR extends the deepcopy mechanism for SoABlocks. Currently deepcopy was not implemented for SoABlocks.

PR validation:

Added tests for nested blocks and deepcopy of SoABlocks

Tests passed

FYI @sbaldu @felicepantaleo

cmsbuild · 2026-07-24T13:01:31Z

cms-bot internal usage

cmsbuild · 2026-07-24T13:03:34Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51554/50290

There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/Portable/test/BuildFile.xml modified in PR(s): SoAMultiView #51458
- File DataFormats/SoATemplate/interface/SoABlocks.h modified in PR(s): Implement heterogeneous Association Map based on SoA blocks #49995
- File DataFormats/SoATemplate/interface/SoACommon.h modified in PR(s): Various fixes for heterogeneous utilities #47605, SoA Schema evolution #50691, Remove explicit implementation of SoAConstParametersImpl #51528
- File DataFormats/SoATemplate/interface/SoALayout.h modified in PR(s): Various fixes for heterogeneous utilities #47605, [Draft] Allow users to set a valid data range for the SoA View #49204, SoA Schema evolution #50691
- File DataFormats/SoATemplate/test/BuildFile.xml modified in PR(s): SoAMultiView #51458

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51554/50290/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51554/50290/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

Co-authored-by: Leonardo Beltrame <leonardobeltrame00@gmail.com>

cmsbuild · 2026-07-24T13:11:15Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51554/50291

There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/Portable/test/BuildFile.xml modified in PR(s): SoAMultiView #51458
- File DataFormats/SoATemplate/interface/SoABlocks.h modified in PR(s): Implement heterogeneous Association Map based on SoA blocks #49995
- File DataFormats/SoATemplate/interface/SoACommon.h modified in PR(s): Various fixes for heterogeneous utilities #47605, SoA Schema evolution #50691, Remove explicit implementation of SoAConstParametersImpl #51528
- File DataFormats/SoATemplate/interface/SoALayout.h modified in PR(s): Various fixes for heterogeneous utilities #47605, [Draft] Allow users to set a valid data range for the SoA View #49204, SoA Schema evolution #50691
- File DataFormats/SoATemplate/test/BuildFile.xml modified in PR(s): SoAMultiView #51458

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51554/50291/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51554/50291/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2026-07-24T14:55:12Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51554/50294

There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/Portable/test/BuildFile.xml modified in PR(s): SoAMultiView #51458
- File DataFormats/SoATemplate/interface/SoABlocks.h modified in PR(s): Implement heterogeneous Association Map based on SoA blocks #49995
- File DataFormats/SoATemplate/interface/SoACommon.h modified in PR(s): Various fixes for heterogeneous utilities #47605, SoA Schema evolution #50691, Remove explicit implementation of SoAConstParametersImpl #51528
- File DataFormats/SoATemplate/interface/SoALayout.h modified in PR(s): Various fixes for heterogeneous utilities #47605, [Draft] Allow users to set a valid data range for the SoA View #49204, SoA Schema evolution #50691
- File DataFormats/SoATemplate/test/BuildFile.xml modified in PR(s): SoAMultiView #51458

cmsbuild · 2026-07-24T14:55:33Z

A new Pull Request was created by @Electricks94 for master.

It involves the following packages:

DataFormats/Portable (heterogeneous)
DataFormats/SoATemplate (heterogeneous)

@cmsbuild, @fwyzard, @makortel can you please review it and eventually sign? Thanks.
@missirol, @mmusich, @rovere this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

makortel · 2026-07-24T15:00:34Z

enable gpu

makortel · 2026-07-24T15:00:39Z

@cmsbuild, please test

cmsbuild · 2026-07-31T13:15:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51554/50402

There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/Portable/test/BuildFile.xml modified in PR(s): SoAMultiView #51458
- File DataFormats/SoATemplate/interface/SoABlocks.h modified in PR(s): Implement heterogeneous Association Map based on SoA blocks #49995
- File DataFormats/SoATemplate/interface/SoACommon.h modified in PR(s): Various fixes for heterogeneous utilities #47605, SoA Schema evolution #50691, Remove explicit implementation of SoAConstParametersImpl #51528
- File DataFormats/SoATemplate/interface/SoALayout.h modified in PR(s): Various fixes for heterogeneous utilities #47605, [Draft] Allow users to set a valid data range for the SoA View #49204, SoA Schema evolution #50691
- File DataFormats/SoATemplate/test/BuildFile.xml modified in PR(s): SoAMultiView #51458

cmsbuild · 2026-07-31T13:15:48Z

Pull request #51554 was updated. @cmsbuild, @fwyzard, @makortel can you please check and sign again.

Electricks94 · 2026-07-31T13:16:03Z

If I understand the code correctly, that's the deep copy of a SoA with a Block.

Is there a test for a SoA with a nested Block, i.e. a Block that contains another Block ?

I extended the nested blocks deep copy test case. It revealed a logical error in the recursive implementation of the blocks deep copy. I fixed it and updated the PR

fwyzard · 2026-07-31T17:38:15Z

😇

fwyzard · 2026-07-31T17:38:23Z

please test

cmsbuild · 2026-07-31T19:55:49Z

-1

Failed Tests: amd_mi300xUnitTests amd_w7900UnitTests nvidia_h100UnitTests nvidia_l40sUnitTests nvidia_t4UnitTests
Size: This PR adds an extra 32KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0ac7d4/55035/summary.html
COMMIT: b8fbf4d
CMSSW: CMSSW_20_1_X_2026-07-31-1100/el9_amd64_gcc13
Additional Tests: GPU,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51554/55035/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3588680
DQMHistoTests: Total failures: 4
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3588658
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files
TriggerResults: no differences found

AMD_MI300X Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 57 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 172855
DQMHistoTests: Total failures: 7460
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 165395
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_H100 Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 62 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 172855
DQMHistoTests: Total failures: 8039
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 164816
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_L40S Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 84 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 172855
DQMHistoTests: Total failures: 8238
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 164617
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_T4 Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 36 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 172855
DQMHistoTests: Total failures: 6934
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 165921
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

Max Memory Comparisons exceeding threshold NVIDIA_T4

@cms-sw/core-l2 , I found 1 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...

Error: Workflow 34434.7503_TTbar_14TeV+Run4D121_HLTHeterogeneousValid step2 max memory diff -36.5 exceeds +/- 30.0 MiB

fwyzard · 2026-07-31T21:40:25Z

ignore tests-rejected with ib-failure

fwyzard · 2026-07-31T21:40:45Z

+heterogeneous

cmsbuild · 2026-07-31T21:40:49Z

This pull request is fully signed and it will be integrated in one of the next master IBs (test failures were overridden). This pull request will now be reviewed by the release team before it's merged. @ftenchini, @mandrenguyen, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2026-08-02T07:27:51Z

+1

Implemented support for nested blocks layouts

42d8917

cmsbuild added this to the CMSSW_20_1_X milestone Jul 24, 2026

cmsbuild added pending-signatures tests-pending orp-pending code-checks-pending heterogeneous-pending labels Jul 24, 2026

cmsbuild added code-checks-rejected and removed code-checks-pending labels Jul 24, 2026

Electricks94 force-pushed the nestedBlocks branch from 6079dd1 to cc67220 Compare July 24, 2026 13:08

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jul 24, 2026

Added nested block deep copy

28b8cdc

Co-authored-by: Leonardo Beltrame <leonardobeltrame00@gmail.com>

Electricks94 force-pushed the nestedBlocks branch from cc67220 to 28b8cdc Compare July 24, 2026 13:10

cmsbuild added code-checks-rejected and removed code-checks-pending labels Jul 24, 2026

Improve deep copy test case for SoABlocks

681ebe8

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jul 24, 2026

Electricks94 changed the title ~~Allows nested Blocks in SoABlocks~~ Nested SoABlocks Jul 24, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 24, 2026

cmsbuild removed the tests-pending label Jul 24, 2026

cmsbuild added tests-pending code-checks-pending and removed code-checks-approved labels Jul 31, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 31, 2026

cmsbuild added tests-started and removed tests-pending labels Jul 31, 2026

cmsbuild added tests-rejected and removed tests-started labels Jul 31, 2026

cmsbuild added fully-signed tests-approved heterogeneous-approved tests-ib-failure and removed pending-signatures tests-rejected heterogeneous-pending labels Jul 31, 2026

cmsbuild added orp-approved and removed orp-pending labels Aug 2, 2026

cmsbuild merged commit 8368712 into cms-sw:master Aug 2, 2026
18 of 24 checks passed

cmsbuild mentioned this pull request Aug 5, 2026

METAlgorithms: Cleanup dependency to avoid linking TF for GenMETProducer #51632

Merged

Uh oh!

Conversation

Electricks94 commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Jul 24, 2026

Uh oh!

cmsbuild commented Jul 24, 2026

Uh oh!

cmsbuild commented Jul 24, 2026

Uh oh!

cmsbuild commented Jul 24, 2026

Uh oh!

makortel commented Jul 24, 2026

Uh oh!

makortel commented Jul 24, 2026

Uh oh!

cmsbuild commented Jul 31, 2026

Uh oh!

cmsbuild commented Jul 31, 2026

Uh oh!

Electricks94 commented Jul 31, 2026

Uh oh!

fwyzard commented Jul 31, 2026

Uh oh!

fwyzard commented Jul 31, 2026

Uh oh!

cmsbuild commented Jul 31, 2026

Comparison Summary

AMD_MI300X Comparison Summary

NVIDIA_H100 Comparison Summary

NVIDIA_L40S Comparison Summary

NVIDIA_T4 Comparison Summary

Max Memory Comparisons exceeding threshold NVIDIA_T4

Uh oh!

fwyzard commented Jul 31, 2026

Uh oh!

fwyzard commented Jul 31, 2026

Uh oh!

cmsbuild commented Jul 31, 2026

Uh oh!

mandrenguyen commented Aug 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Electricks94 commented Jul 24, 2026 •

edited

Loading

cmsbuild commented Jul 24, 2026 •

edited

Loading